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An Apparatus, System and Method for Secure, Recoverable, 
Adaptably Compressed File Transfer 



* TECHNICAL FIELD 

This invention relates in general to the field of file compression and transfer, and 
more particularly to an improved apparatus, system a^d method of achieving secure, 
recoverable, adaptably compressed file transfers. 

1 0 BACKGROUND OF THE IN VENTION 

The ability to move data reliably and securely from one location to another has 
become an important key to success, and in many cases to survival, for many companies 
and businesses. In businesses for which the primary product or service delivered is data, 
' this ability is even more critical. 

Recent years have seen a proliferation of computers and people connected on a 
network, in one form or another. Companies have come to depend on the ability to move 
data on the network to accomplish their tasks and to ensure the continuation of their 
businesses. A wide variety of connections and speeds are inherent on these networks, 
20 ranging from cellular phone modems and dial-up low-speed connections up through 
ISDN lines, Tl dedicated lines, and high speed ATM connections. A wide variety of 
different computers with different processing speeds and communications capabilities 
are attached as nodes on these networks. In many cases, especially in the oil and gas 
industry, many of the computers are mobile and come and go (attach and detach) from 
the networks from different locations. 



25 



The need to transmit information on these widely varying networks has created 
a demand for methods, processes, and standard techniques for moving data from one ' 
computer system to another in a secure, efficient, and reliable manner. Many of the 
30 lower to mid-level protocols have been accepted as standards. Among the most notable 
. of these is the Transport Control Protocol/Internet Protocol (TCP/IP). This is the basic 
method for moving packets of data on standard networks, including the Internet. 

While there have been various applications developed using TCP/IP for 
35 transmission, the most widely known is the File Transfer Protocol ("FTP"). FTP was 
developed to support many different hardware and operating system platforms and is 
widely used to move data files around networks. It works fairly well in well established, 
reliable networks but has some limitations on noisy, unreliable, very low bandwidth 
network connections. Another disadvantage of FTP is that FTP does no compression of 
40 data, so all data must be transmitted as is. FTP also provides no recovery mechanism 
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for file transfers. This means that if the connection is lost during a file transmission, the 
file transfer must be restarted from the beginning. FTP has no inherent security 
mechanism for protecting the data "on the wire" during the transmission. FTP also 
requires that the all data to be transmitted is available, i.e. that the complete file is 
5 available, before transmission can begin. Yet another disadvantage of FTP is that FTP 
requires that the file is not being written when the transmission begins or is in progress. 

One of the many needs for secure, recoverable, adaptively compressed file 
transfers may be found in the oil and gas industry. In the oil and gas industry, operating 
1 o companies which own and/or manage hydrocarbon wells evaluate the wells by wireline 
logging. In wireline well logging, one or more tools are connected to a power and data 
transmission cable or "wireline" and are lowered into the well borehole to obtain 
measurements of geophysical properties for the area surrounding the borehole. The 
wireline supports the tools as they are lowered into the borehole, supplies power to the 
15 tools and provides a communication medium to send signals to the tools and receive 
data from the tools. Commonly, tools are lowered to a depth of interest in the well and 
are then retrieved. As the tools are retrieved, they send data about the geological 
formations through which they pass through the wireline to data acquisition and 
processing equipment at the surface, usually contained inside a logging truck or a 
logging unit. 

The data acquisition and processing equipment, including software, compiles 
the data from the tools into a "log," a plot which presents the geophysical information 
concerning the geological formations encountered by the well, frequently by depth. 
25 Logs can also be used to evaluate current production from producing wells or to inspect 
the integrity of production equipment in a producing well. In any case, the data gathered 
during the logging operation is generally presented on the log by depth, but may also be 
presented by time, or any other index by which multiple physical entries are recorded. 
U.S. Patent No. 5,051,962 (incorporated by reference) describes such a well logging 
system controlled by a general purpose computer programmed for real time operation. 
Various data acquisition and processing software programs are known in the art. An 
example of data acquisition and processing software is Schlumberger's proprietary 
MAXIS™ system, which is a suite of separate computer programs. 

35 The data acquisition and processing software writes the log data to two types of 

locked format files on disk. By "locked," it is meant that the format files cannot be 
written to and read from at the same time. The two types of locked format files are 
distinguished by the type of information they contain: one is a data format file and the 
other is a graphics format file. The data format file contains the numerical properties of 

40 the log data; the graphics format file contains the pictorial representation of the data. 
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The data acquisition and processing software continues writing the log data to the 
locked data format file and the locked graphics format file until the log is complete. 
Then the data from the locked data format file and the locked graphics format file may 
be translated from digital readings into physical form by a marking device such as a 
5 printer. In addition to the locked data format file and the locked graphics format file, the 
data acquisition and processing software may send the log data to a viewing monitor, 
via a renderer. Using the monitor, the well logging professional ("logging engineer") 
conducting the logging operation can view the log as it is being compiled. 

10 After the log is compiled, it may be transmitted to the operating company's 

headquarters for interpretation and review by management. The paper log may be sent 
directly from the wellsite to the operating company as a facsimile. Alternatively, the 
completed locked data format file may be sent from the wellsite to a data processing 
center via satellite using a protocol such as DECNET. The data processing center could 

15 in turn transmit the log as a facsimile to the operating company. As another alternative, 
the completed locked data format file may be sent from the wellsite to an operating 
company using a computer program such as Blast™ by U.S. Robotics. 

The data acquired by logging is often crucial to the decision-making process on 
20 what will be done with the well being logged. Take, for example, a well which has just 
been drilled and logged. Depending on the results of the log, the well could be drilled 
deeper, plugged and abandoned as non-productive or cased and tested — or perhaps the 
decision will be that additional logs are required before the decision on the disposition 
of the well can be made. The results of the log may also help determine whether the well 
25 requires stimulation or special completion techniques, such as gas lift or sand control. 
In any case, these decisions are critical and have to be made very quickly. Mistakes or 
even mere delay can be extremely expensive. 

Because log interpretation is part art and part science, the operating company 
2Q which is drilling or producing the well frequently desires to have its own personnel 
viewing the log data as the well is being logged. But the operating company may be 
located half a world away from the well itself. Drilling and production activities are 
often located in remote locations and it is difficult for the operating company to have its 
own personnel, such as a geologist or petrophycist, join the wireline company's logging 
35 engineer on site during the logging operation. Sometimes logistics or severe weather 
conditions prevent the operating company from sending anyone to the wellsite for the 
logging operation. Furthermore, sending personnel to wellsites is expensive and 
exposes them to all of the hazards of the drilling or production operation, as well as the 
hazards and inconvenience of travel. As a consequence, tentative decisions often have 

40 
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to be made before the operating company can complete its own review of the actual 
logging data, relying solely on the interpretations conducted at the wellsite. 

Accordingly, a need exists for a system or method which would allow files to be 
transferred securely, especially while "on the line." 

A further need exists for a system or method which would allow a file to be 
transferred while making maximum use of low bandwidth connections. 

A further need exists for a system or method which would allow a file to be 
transferred while adaptably compressing the file to improve transmission throughput. 

A further need exists for a system or method which would overcome the 
disadvantages of the File Transfer Protocol. 

A further need exists for a system or method which would allow files to be 
transferred taking into account the unique requirements of mobile network connections. 

A further a need exists for a system or method which would allow files to be 
transferred as they are compiled in at least near real time from one location to a remote 
location remote from the primary for viewing or other use. 

A further need exists for a system or method which would allow well data files 
to be transferred as they are compiled in at least near real time from a wellsite to a 
remote location remote from the well site for viewing or other use. 

A further need exists for a system for or method of file transfer which would 
provide a recovery method should communications be lost. 

Because the data from the logging operation is of a highly competitive nature 
and is extremely confidential, a need exists for a system or method which will send well 
data files from a wellsite to a remote location in near real time, in such a way that the 
data files are not susceptible to being misdirected or lost. 

30 A further need exists for a system or method which can maintain the 

confidentiality of the well data while it is being transmitted. 

A further need exists for a system or method of transferring files in near real 
time from one location to a remote location so that so that persons can view the files in 
near real time, without the expense of travelling to the primary location. 

A further need exists for a system or method of transferring well data files in 
near real time from wellsite to a remote location remote from the wellsite so that persons 
can view the well data files near real time as they are being compiled, without the 
expense of travelling to the wellsite and without being exposed to the hazards of the 
40 wellsite. 
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SUMMARY OF THE INVENTION 

In accordance with the present invention, an apparatus, system and method are 
provided that substantially eliminate or reduce the disadvantages and problems 
associated with the previously developed file transfer systems. 

5 

The present invention provides a system for handling and transmitting a file 
over a communication channel which includes a file having a degree of compressibility; 
a communications channel having a physical bandwidth; a file transfer server at a first 
location; a file transfer client at a second location and a means for compressing the file 
10 based on the physical bandwidth, the capabilities of the transmitting and receiving 
processors and the degree of compressibility of the file. 

The present invention also provides for a system for managing and transmitting 
a file over a communication channel which includes a file having a degree of 
compressibility and a plurality of buffers, a communications channel having a physical 
bandwidth;, a file transfer server at a first location, a file transfer client at a second 
location, a feedback loop for optimally compressing the file for transmission, including 
a means for compressing a first buffer to a first level of compressibility; a means for 
evaluating the efficiency of the compression of the first buffer, a means for adjusting 
20 the compression of second buffer based on the evaluation of the compression of the first 
buffer. 

The present invention also provides for a system for managing an^^ansmittir 
a file over a communication channel which includes a file and a means foi^potnpressing 
buffers of the file in a stream in real time while it is being written. 

25 

The present invention also provides for a system for managing and transmitting 
a file over a communication channel which includes a file, a transmission of the file 
having a state and a location, and a means for maintaining the state and location of the 
transmission within the file so that transmission can be resumed in the event of an 
30 interruption at the point of the interruption. The interrupted transmission may be 
automatically resumed or manually resumed. 

The present invention also provides for a system for managing and transmitting 
a file over a communication channel which includes a file, a means for encrypting the 
25 file in a stream before and during transmission, and means for de-crypting the file after 
receipt of transmission. 

The present invention also provides for a system for managing and transmitting 
a file over a communication channel which includes a file having a degree of 
compressibility and a plurality of buffers, a communications channel having a physical 
^ bandwidth, a transmitting processor, a receiving processor, a means for optimally 
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compressing the file based on the physical bandwidth, the capabilities of the 
transmitting and receiving processors and the degree of compressibility of the file, a 
means for compressing the buffers of the file in a stream in real time while the file is 
being written, a transmission of the file having a state and a location, a means for 
5 maintaining the state and location of the transmission within the file so that 
transmission can be resumed in the event of an interruption at the point of the 
interruption, a means for encrypting the file in a stream- before and during transmission; 
and a means for de-crypting the file after receipt of transmission. 

1 o The present invention also provides for a method of compressing a source file, 

having a plurality of buffers, for transmission over a communication channel, which 
includgs-selexting a fjj^E^^j^o f4&e source file, compressing the first buffer to a first 
co mpression ley fct> marshalling the first buffer, transmitting the first buffer, de- 
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compressing the first buffer, writing the first buffer to a destination file, determine a 
1 5 first throughput which was used for steps (a) through (f), selecting the compression 
level of a sepondi5uffSFba^fed on the first throughput, repeating steps (a) through (h) for 



each ofSh^btrftefsih turn until all of the buffers of the source file have been 
transmittedari^irav^een written to the destination file. The marshalling steps may 
include encrypting the buffers and the de-compressing steps include de-encrypting the 
buffers. There may also be a further step of writing to the source file while performing 
one or more of steps (a) through (i). 

The present invention also provides for a system for handling and transmitting 
a first file over a communication channel which includes a means for reading while 
25 writing a first file, a means of compressing the first file, a means of marshalling the first 
file, a means for transmitting the first file from a firsUocation to a .second location, a 
means for unmarshalling the first file, a means fo£deoemp£gssing.thf: first^file, a means 
for writing the first file to a second file. The means a mean^for reading while writing a 
first file, a means of compressing the first file and means of marshalling the first file 
may comprise a file transfer server. The means for unmarshalling the first file, a means 
for decompressing the first file, and a means for writing the first file to a second file may 
be a file transfer client. The file transfer server may inc lude a read while write module 
and/or a compression module. The^Ger^^ ^sion moduje jrfay use a deflation algorithm 
and or a Huffman tree.The compression module may compress in a first stage to 
produce literals and pointersand may compress in a second stage to produce a Huffman 
tree and a block. The invention may also include a means for encrypting the first file 
before marshalling the first file and a means for decrypting the first file after the first 
file is transmitted. The file transfer client may include a read while write; module. The 
40 first file may be a sharable file. 
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The present invention also provides for a system for handling and transmitting 
a first file over a communication channel which includes a means for reading while 
writing data to the first file, a means for compressing the first file, a means for queueing 
the first file, a means for marshalling the first file, a means for transmitting the first file 
from a first location to a second location, a first means for unmarshalling the first file, 
a means for re-queueing the first file, a means for re-marshalling the first file, a remote 
procedure call for sending the first file within the second location, a second means for 
unmarshalling the first file, a means for decompressing the first file, a means for writing 
the first fileJg_a-Sgcond file. The means for reading while writing data to the first file, 
for com pressing th ejfcst file, for queueing the first file, and for marshalling the first 
Jilemay be a first file transfer server. The means for a first means for unmarshalling the 
first file, a means for re-queueing the first file, a means for re-marshalling the first 
filemay be a file transfer client. The file transfer server may include a read while write 
15 module. The file transfer server may include a compression module.The compression 
module may use a deflation algorithm or aa Huffman tree. The compression module 
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may compress in a first stage to produce literals and pointers. The^grtrfffFssion module, 
may compress in a second stage to produce a Huffman tree ana~a~BIock. The present 
invention may further include a first means for encrypting the first file before 
20 marshalling the first file and a first means for decrypting the first file after the first file 
is transmitted. The present invention may further include a second means for encrypting 
the first file after the first file is transmitted and a a first means for decrypting the first 
file after the first file has been sent through the remote procedure call within the second 
location. The means for a second means for unmarshalling the first file, a means for 
decompressing the first file, a means for writing the first file to a second file may be a 
second file transfer server. The file transfer client may include a read while write 
module. The first file is a sharable file. The second file may be a sharable file. 

The present invention also provides for a method for handling and transmitting 
30 a first file over a communication channel including the steps of reading while writing 
data to the first file, compressing the first file, queueing the first file, marshalling the 
first file, transmitting the first file from a first location to a second location, 
unmarshalling the first file, ^gSgpressing the firstf ih^ and writing the first file to a 
second file. The steps of reading while writing data to the first fil^ompressmgt^ first 
35 file, queueing the first file, marshalling the first file may be accomplished by a file 
transfer server. The steps of unmarshalling the first fiierraecompressirJ^^ie first file, and 
writing the first file to a second file may be accomplished by a tile transfer client. The 
file transfer server may include a read while write module. The file transfer server may 
include a compression module. The^empfession steprrj^y include the use of a deflation 
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algorithm for a first stage of compression or a Huffman tree for a second stage of 
compression. The use of the deflation algorithm in the first stage may produce literals 
and pointers. The use in a second stage of the Huffman tree may produce a Huffman 
tree and a block. The present invention may also include the futher include the steps of 
5 encrypting the first file before marshalling the first file and decrypting the first file after 
the first file is transmitted. The file transfer client may include a read while write 
module. The first file may be a sharable file. The second file may be a sharable file. 

The present invention also provides for a method for handling and transmitting a 
1 o first file over a communication channel including the steps of reading while writing data 
to the first file, compressing the first file, queueing the first file, marshalling the first 
file, transmitting the first file from a first location to a second location, linmarshalling 
the first file, re-queueing the first file, re-marshalling the first file, sending via a remote 
procedure call the first file within the second location, unmarshalling the first file a 
1 5 second time, decompressing the first file, and writing the first file to a second file. The 
steps of reading while writing data to the first file, compressing the first file, queueing 
the first file, marshalling the first file may be accomplished by a first file transfer server. 
The steps unmarshalling the first file, re-queueing the first file, re-marshalling the first 
file may be accomplished by a file transfer client. The file transfer server may include 
a read while write module.The file transfer server includes a compression module. The 
method for handling and transmitting a first file as in claim 52 wherein the compression 
step includes use of a deflation algorithm for a first stage of compression. The 
compression step uses a Huffman tree for a second stage of compression. The use of the 
Huffman tree produces a Huffman tree and a block.T he use of the deflation algorithm 
in the first stage produces literals and pointers.The present invenrtion may also futher 
comprise the steps of encrypting the first file before marshalling the first file, decrypting 
the first file after the first file is transmitted to the second location; 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 illustrates a primary location in communication with a remote location according 
to the present invention. 

5 Fig. 2 illustrates the data acquisition and processing equipment of the primary location, 
including inputs and outputs. 

Fig. 3 illustrates the remote location equipment, including inputs and outputs. 
. j Fig. 4 A illustrates the components of the primary memory. 
Fig. 4B illustrates the components of the remote memory. - 

FlG. 5A illustrates the data acquisition and processing software and other software 
programs at the primary location and the file transfer to the remote location. FlG. 5B 
15 illustrates the the remote data processing software and other software programs at the 
remote location and the file transfer to the primaty location. 

Fig. 6 illustrates the transmission process if near real time display is not desired. 

20 F JG - 7A and Fig. 7B illustrate the transmission process if near real time display is 
desired. 

Fig. 8 illustrates the communication system. 
^ . FlG. 9 illustrates the compression details. 
Fig. 10 illustrates adaptable compression. 
Fig. 11 A - 11F illustrate the different scenarios for file transfer. 
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DESCRIPTION OF A PREFERRED EMBODIMENT 

The above-noted and other aspects of the present invention will become more 
apparent from a description of a preferred embodiment, when read in conjunction with 
the accompanying drawings. The drawings illustrate the preferred embodiment of the 
5 invention. In the drawings, the same members have the same reference numerals. 

I. Overview 

As described in co-pending U.S. Patent Application Serial Number and as 

illustrated in FiG.l, during logging operations, log data is sent from a logging tool 10 

10 through wireline 20 to a data acquisition and processing system 30 at a wellsite or 
primary location 40. ("Primary is used in the sense of the word "first/' not "most 
important.") A primary input device 50, such as a keyboard, allows human input into 
the data acquisition and processing system 30. Outputs to the data acquisition and 
processing system 30 include a primary monitor 60, and a primary printer 70, which 
acts through a primary operating system device driver 80 to print a log 90. The data 
acquisition and processing system 30 communicates with a remote location 100 through 
a communication channel 110. The remote location 100 has a remote location 
equipment 120, a remote input device 130 for human input, such as a keyboard, and 

20 outputs, such as a remote monitor 140 and a remote printer 150 which acts through a 
remote operation system device driver 160 to produce a log identical to the log 90 
produced at the primary location. (Although the logs have separate physical existences, 
both logs are given the reference number 90 to indicate this identity.) 

Continuing to refer to Fig. 1, the data acquisition and processing equipment 30 

25 ' ■ • 

and the remote location equipment 120 establish communfcatron — over tKe~ 

communication system 110. Preferably, the communication system 110 uses TCP/IP 

protocol and is based on a physical link, such as hard-wir ^^llui^3:adio,^ icrowave v 

satellite or telephone transmission. In alternative embodiments, other types of point-to- 

30 point network protocols, such as Blast by U.S. Robotics, may be used. In still other 
embodiments, other types of communication systems could be used. If using the TCP/IP 
protocol, both the data acquisition and processing equipment 30 and remote location 
equipment 120 have individual IP addresses 102, 104 in accordance with the TCP/IP 
protocol. The bandwidth of the communication system 110 is preferably at least 10 

35 kilobits per second. As illustrated in Fig. 8, the components of the communication 
system 110 include a physical link 111, the protocol 112 such as TCP/IP, and the 
transport mechanism 113, as well as other components illustrated. 

Fig. 2 illustrates the data acquisition and processing equipment present at the 
primary location 40, including its inputs and outputs. As illustrated in FlG. 2, the. log 
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data enters the data acquisition and processing equipment 30 through a data acquisition 
system 170. The data acquisition and processing equipment 30 also includes a primary 
data processor 180, a primary bus 190, a primary file system 195, and a primary 
memory 200. The primary input device 50, the primary monitor 60, the primary printer 
70, the primary operating system device driver 80, and the log 90 are also illustrated. 

Fig. 3 illustrates the remote location equipment 120, including its inputs and 
outputs. As illustrated in FlG. 3, the remote location equipment 120 includes a remote 
data processor 210, a remote bus 220, a remote file system 225, and a remote memory 
10 230. The remote input device 130, the remote monitor 140, the remote printer 150, the 
remote operating system device driver 160, and the log 90 are. also illustrated. The 
remote data processor 210 is preferably a Pentium PC (P5-90 or higher), with an 
Ethernet interface or a modem. The remote memory 230 has preferably at least 32 
Megabyte RAM. 

FlG. 4 A illustrates the components of the primary memory 200. As illustrated in 
FlG. 4 A, the primary memory 200 includes a primary data acquisition and pr ocessi ng 
software 240, a primary operating system software 250, a primary da ^Tstorage^26 fl^k 
primary file transfer utility program 270 and other primary software 280. The primary 
operating system software 250 is preferably Windows® NT™ by Microsoft 
Corporation. If near real time viewing is not desired, Open VMS or other operating 
systems may be used. 

FlG. 4B illustrates the components of the remote memory. As illustrated in FlG. 
4B, the remote memory 230 includes a remote data processing software 290, a remote 
operating system software 300, a remote data storage 310, a remote file transfer utility 
program 320 and other remote software 330. The remote. operating system software 300 
is preferably Windows® NT™ by Microsoft Corporation. 

FlG. 5A illustrates the data acquisition and processing software 240 and other 
30 software programs at the primary location and FlG. 5B illustrates the remote data 
processing software 290 and other software programs at the remote location. As 
illustrated by FlG. 5A, as log data enters the data acquisition and processing software 
240, the data enters a primary data formatting program 340 where it is formatted as 
numerical data in industry standard format for storage. The data also enters a log 
generating program 350 which adds commands and other instructions to the data to 
create graphics data. The numerical data includes numerical properties of the data; the 
graphics data includes pictorial representation of the data. 

The data formatting program 340 and the log generating program 350 act 
40 through a primary read -while- write ("RWW") module 360 to write the data as it is 
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received to a primary sharable data format file 370 and to one or more primary sharable 
log graphics files 380, respectively. (Different primary sharable log graphics files 380 
may be created for different presentations of the same log data.) 

Unlike a locked format file, a sharable format file allows shared file access, such 
that the sharable format file may be written to arid read from at the same time. 
Preferably, each sharable format file contains all of the acquired data and input 
parameters for a single logging run, that is, for the data from a set of logging tools run 
into the well simultaneously. 

As the primary RWW module 360 begins to write logging data to the primary 
sharable data format file 370 and the primary sharable graphics format file 380, the 
primary RWW module 360 also creates for each of the primary sharable data format file 
370 and for the primary sharable graphics format file, 380, a primary semaphore file, 
390, 400 respectively. The primary semaphore files 390, 400 have the same name as the 
primary sharable format file for which they were created, with the addition of a "__smf * 
at the end. The existence of a semaphore file indicates that the primary sharable format 
file with the similar name is a sharable file. Any program which shares access to the 
sharable format file also uses a copy of the primary RWW module to check for the 
2Q existence of the associated semaphore file to determine whether the format file is 
sharable. 

Refering to FlG. 5A, the primary file transfer utility program 270 may include a 
primary file transfer server 410 (also illustrated in FlG. 5B) and a primary file transfer 
client 420. Refering to Fig. 5B, the remote file transfer utility program 320 may include 
a remote file transfer server 430 and/or a remote file transfer client 440. The minimum 
needed is a file transfer server in either the primary or remote location and a file transfer 
client in the other location. But there may be a file transfer server and a file transfer 
client in each location. 

30 The primary file transfer server and the remote file transfer server are always on 

and are always listening for requests from one of the file transfer clients. The file 
transfer clients are not always on, but are send requests or queries to the file transfer 
server as initiated by a request from the user through the input device. One file transfer 
server may handle requests from more than one file transfer client. File transfer clients 

35 may or may not read from or write to one of the format files. A file transfer server reads 
from and writes to a format file. Different scenarios for data requests between file 
transfer clients and servers are found in FlG. 11. 
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After establishing communication, the remote file transfer client 440 requests 
data, either numerical data, graphics data or both, from the primary file transfer server 
410. Depending on which data it is seeking, the primary file transfer server 410, which 
contains a copy of the primary RWW module 360, checks for primary semaphore files 
390, 400 for both or either of the primary sharable data format file 370 and the primary 
sharable graphics format file 380. (For example, the primary file transfer server 410, 
. . using the primary RWW module 360, would check for N primary semaphore file 390 for 
the primary sharable data format file 370.) After verifying that the primary sharable data 
. format file 370 and/or the primary sharable graphics format file 380 are sharable, the 
primary file transfer server 410, using its copy of the primary RWW module 360, begins 
to read the data written the primary sharable data format file 370 and/or the primary 
sharable graphics format file 380. The primary file transfer server 410 starts at the 
beginning of the data in the primary sharable data format file 370 and/or the primary 
sharable graphics format file 380 and continues to read a certain amount of data (as 
described below) at a time until it reaches a last datum written. Then the primary file 
transfer server 410 continues to read additional data as it is being written to the primary 
sharable data format file 370 and/or the primary sharable graphics format file 380 until 
the logging operation is completed. At that time, the primary sharable data format file 
370 and the primary sharable graphics format file 380 are closed by the data acquisition 
and processing software 240. The data acquisition and processing software 240 also 
closes and deletes the similarly-named semaphore files 390, 400. 

III. Compression and Decompression 

The primary file transfer server 410 reads approximately 32,000 bytes (or 32 
kilobytes) of data at a time into a buffer, which is then compressed. The compression 
and decompression functions are performed by a "compression module" and a 
"decompression module." The compression module and the decompression module use 
variations of the deflation/inflation algorithms as described in Ziv J., Lempel A., "A 
Universal Algorithm for Sequential Data Compression", IEEE Transactions on 
Information Theory, Vol. 23, No. 3, pp. 337-343 (1977) ("LZ77"), incorporated herein 
by reference. 

As described below, compression is accomplished on two stages. 
A. Detailed Description of Compression: First Stage 
1. The Deflation Algorithm 
a. Overview 
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As illustrated in FlG. 9, the compression module uses a deflation algorithm 450 
which is a variation of that described in LZ77. The deflation algorithm 450 finds 
duplicated strings of data in the input data using a fixed-length sliding window. In this 
description, "string" must be taken as an arbitrary sequence of bytes, arid is not 

5 restricted to printable (ASCII) characters. Preferably, the string is at least three bytes 
long. As the fixed length sliding window slides along the data, it looks for re- 
occurrences of a string of data. The re-occurrence of the string is replaced by a pointer 
to the previous string. The pointer is expressed as a pair of numbers representing 

■ distance arid length. Distance means the distance from the beginning of one string and 
the beginning of its duplicate string. Length is the length of the string. Preferably, 
distances are limited to 32K bytes, while lengths are limited to 258 bytes. When a string 
does not occur anywhere in the previous 32K bytes, the string is emitted as a sequence 
of literal bytes. A sequence of literal bytes is not compressible. 

15 In still greater detail, the deflation algorithm actually finds the duplicate strings 

using a hash table. As the fixed length sliding window slides over the data, all input 
strings of at least three bytes in length are inserted into the hash table. An entry into the 
hash table is called a hash chain. The deflation algorithm computes a hash index, which 

points to the location of the hash chain, for the next three bytes. If the hash chain for this 

20 . 

index is not empty, all strings in the chain are compared with the current input string, 

and the longest match is selected. 

The hash chains are searched starting with the most recent strings, to favor small 
distances and to take advantage of Huffman encoding, described below. The hash 
25 chains are singly linked. There are no deletions from the hash chains; the deflation 
algorithm simply discards matches that are too old. 

To avoid a worst-case situation, very long hash chains are arbitrarily truncated 
at a certain length, determined by a runtime option. That is, one may select a level of 

3Q compression, preferably Levels 1 to 9, to determine at what length to truncate the hash 
chain. Each level will discard hash chains of specific lengths. Because the very long 
hash chains may be truncated, the compression module does not always find the longest 
possible match. The compression module generally finds, however, a match which is 
long enough. Level 0 implements another algorithm (a "stored algorithm") which 

35 consists of only writing header information into the output buffer and then copying 
input to output unchanged, i.e. there is no compression. 

b. Lazy Evaluation Method 

The compression module may also defer the selection of the matches with a lazy 
40 evaluation mechanism. The lazy evaluation mechanism is preferably used for Levels 4 
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and higher. After a match of length N has been found, the compression searches for a 
longer match at the next input byte. If a longer match is found, the previous match is 
truncated to a length of one (thus producing a single literal byte) and the longer match 
is emitted afterwards. Otherwise, the original match is kept, and the next match search 
5 is attempted only N steps later. 

The lazy match evaluation is also subject to a runtime parameter. If the current 
match is long enough, the compression reduces the search for a longer match, thus 
speeding up the whole process. (See the discussion of improving throughput, below.) If 
1 o compression ratio is more important than speed, the compression module attempts a 
complete remote search even if the primary match is already long enough. 

c. Alternative to Lazy Evaluation Method 

The lazy match evaluation is not performed for the fastest compression modes 
] 5 (Levels 1 to 3). For these fast modes, new strings are inserted in the hash table only 
when no match is found, or when the match is not too long. This degrades the 
compression ratio but saves time since there are both fewer insertions and fewer 
searches. 

^ B. Detailed Description of Compression: Second Stage 

Continuing to refer to Fig. 9, a plurality of Huffman trees 460 are used to 
compress the data to still a further stage. Once this stage has been reached, the data 
appears as a stream of pointers and literals. In this stage, the compression module looks 
for repetitions in this stream of pointers and literals. Within the pointers, there may be 
25 a match of lengths ("match lengths") or of distances ("match distances"). Literals and 
match lengths are compressed with a first Huffman tree, and match distances are 
compressed with a second tree. The compression subroutine deals with a block of the 
data stream at a time. The trees are stored in a compact form at the start of each block. 
A block can have any size, except that the compressed data for one block must fit in 
available memory. The block is terminated when the compression module determines 
that it would be useful to start another block with fresh trees. (This is somewhat similar 
to "Compress" and "Zip" programs by Unix.) 

The compression module was designed to allow single pass compression 
35 without any backwards seek, and without a prior knowledge of the uncompressed input 
size or the available size on the output media. If the compression method is 0 (stored) 
however, the stored data comes after the "crc" and "compressed size." This enables fast 
"decompression." 
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Compression is always performed, even if the compressed data is slightly larger 
than the original. This may occur when the data within 32 kilobytes is not compressible 
or if the compression module has been set to Level 0 to turn off the compression. The 
worst case expansion is a few bytes for the compression module buffer header, plus 5 
5 bytes every 32K block, or an expansion ratio of 0.015% for large data. The actual 
number of used disk blocks almost never increases. 

C Decompression \ 

Information needed for the decompression is inserted at the head of the 
1° compressed data. The decompression module simply reverses the Huffman tree 
compression, which results in a sequence of literal bytes and pointers. The 
decompression module replicates the strings pointed to by the distance/length pointers. 

IV. Improving Throughput: Adaptable Compression 

Fig. 10 illustrates the process of adaptive file compression. 

Several factors can affect the throughput of a file transfer. These factors include 
varying bandwidth or delay, CPU availability on the sending or receiving processors, 
and compressibility of the data. The compression is accomplished in such a way as to 
20 optimize the average throughput of the file transfer by modifying compression levels 
for varying conditions. 

The variables involved in total transmission time and their relationships is 
shown by the formula: 

EQ.1:T = C + M + X + D 

where T is the total time required to transmit a buffer of data, C is the time 
required to the compression module the data, M is the marshalling time, that is the time 
required to reorder the data to transmission byte order and to encrypt if desired, X is the 
time required to transmit the data and D is the time required to decompress the data. 

The variables of M and D are small compared to C and X. C and D are both 
proportional to the amount of data transmitted, but C is much larger than D. X is 
proportional to the amount of data transmitted and is inversely proportional to the 
bandwidth of the communication channel. The delay and pipeline size of the 
35 communication channels are also significant factors. 

For instance, a satellite bandwidth may be 256 kilobytes per second and its delay 
1 .25 seconds per round trip. When the data is sent and an acknowledgment is received, 
that is one round trip. The amount of data which may be sent through the pipe at one 

40 
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time ("pipeline size") may be about 28 kilobytes. When the DCE software is installed, 
the pipeline size should be set to 28 kilobytes. 

For very large data files which may consist of hundreds or thousands of buffers, 
it is advantageous to attempt to minimize T. The time to compress, C, can be reduced 
by using a lower level of compression, but this also results in more data to be 
transmitted, which in turn increases X. The relationship between C and X can be stated 
as: 

X = A/C 

10 

where A is a data dependent factor and depends on the bandwidth and the 
compression level. Assuming that M and D are negligible, the formula for total time 
becomes: 

T = C + A/C 

1 5 Attempting the optimal compression level to minimize T would be very difficult 

and the time required to determine that value would also increase T. Since the 
compressibility of the data and the available CPU cycles at both ends if the transmission 
are also major factors, it is not practicable to determine the optimal compression level 

2Q for each buffer. 

However, one can vary the compression level and see whether the throughput 

improves or degrades. If throughput improves, one can change the compression level in 

the same direction as the previous time. If throughput degrades, one can change the 

compression level in the opposite direction. 
25 ' ^ 

For example, if a change in the compression level from 4 to 5 improves 

throughput, then one can increase the compression level still further to 6. If the 

compression level of 6 then decreases throughput, the compression level can be set back 

to 5 for the next attempt. 

Continual variation of the compression level is unlikely to achieve optimized 
throughput for any single transmission. However, continual variation of the 
compression level is likely to yield a better average throughput for a wide range of files 
over varying conditions than are achieved with a fixed compression level. 

35 Reffering to Fig. 10, the Level is set 700 to A, preferably a number between 0 

and 9. 32 kilobytes of data are compressed 710 at Level A. Throuput Tl is calculated 
720 for Level A. Level A is set 730 to A new , while keeping A within Levels o to 9. The 
next 32 kilobytes are compressewd 740 at Level A new and the throughpput at Level 
A nevv , T new is determined750. T new is compared to Tl 760. 

40 
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If T new is greater than Tl, then Tl is set 800 to T new and A new is compared 810 
to A. If A new is greater than A, A is set equal 820 to A plus 1 . If A nevv is not greater than 
A, A is set equal 790 to A minus 1. The steps repeat, starting with step 730. 

If T nevv is not greater than Tl, then Tl is set 800 to T new and A nevv is compared 
780 to A. If A new is less than than A, A is set equal 820 to A plus 1 . If A nevv is not less 
than A, A is set equal 790 to A minus 1 . The steps repeat, starting with step 730. 

V. Encryption, Marshalling and Transmission 

Referring to Fig. 6, Fig. 7a, and FlG. 7B, the compressed buffer is put into a 
queue 540. The compressed buffer may be encrypted, if desired, and marshalled by an 
encryption and marshalling module 550 such as Distributed Computing Environment 
("DCE") by Open Group. Operating systems often order data either "big endian" or 
"little endian" (high order byte or low order byte). For example, VMS and Windows NT 
15 use little endian; UNIX uses big endian. Marshalling means ordering the data into a 
platform independent byte order, so that it can be properly received and understood 
without regard to how the operating system organized it. After marshalling, the buffer 
is transmitted to the remote file transfer client 440 via a remote procedure call 570, such 
as a DCE pipe. The pipe 570 includes a callback mechanism which provides for a 
20 continuous read-send loop. That is to say that the primary file transfer server 410 can 
send a first buffer via the remote procedure call 140 and then read a second buffer of 
data and send it too, without waiting for the remote file transfer client 440 to receive the 
first buffer and request another. Other remote procedure calls in alternative 
embodiments of the invention do not include the callback mechanism. 

25 

If there should be an interruption during transmission of the file, the system 
attempts an automatic recovery. The remote file transfer client 440 will use an 
automatic callback system three times to attempt to re-establish communications. If 
communication is successfully re-established, the transfer of data will resume where it 

30 left off. This is accomplished by the remote file transfer client 440 knowing how many 
bytes of data it has and telling the primary file transfer server 410 to pick up at the next 
byte. If the remote file transfer client 440 has 1000 bytes of data, it will tell the primary 
file transfer server 410 to begin transmitting at byte number 1001. If the attempts are 
unsuccessful, i.e. the communications failure is complete, the transfer can be re-started 

35 manually, in recover mode, to resume from the point of interruption. 

VL At Remote Location 

At the remote location, the transmitted data is de-crypted, unmarshalled, 
decompressed and written to disk. This may be accomplished in many ways as detailed 

40 
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in the next section, but is preferably accomplished in one of two ways, depending on 
whether the data is to be viewed in near real time. 

A. Without Near Real Time Viewing 

If it is not desired to view the data in near real time, it is preferable to allow the 
remote file transfer client 440 to write data directly to the sharable data and log format 
files, instead of using the remote file transfer server to do so. In fact, in such a situation, 
a remote file transfer server is not required. In additidn, in such a case, the remote 
procedure call preferred is a DCE pipe. 

As illustrated in Fig. 6, in such a case, the transmitted data is received by the 
remote file transfer client 440, de-crypted and unmarshalled using a de-cryption/un- 
marshalling module 580. The remote file transfer client 440 then decompresses the 
transmitted data with th£ ^conipfession modu!e"59(Pl &nd uses its own copy of the 
15 RWW module 490 to write the data to the sharable remote data format file 450 and/or 
the sharable remote log format file 460 and create the semaphore files, 470, 480. After 
the data storage is completed, the data may be rendered into a log and printed or viewed 
on the remote monitor. 



TO 
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B. With Near Real Time Viewing 

If the data is to be viewed in near real time, it is preferable to allow the remote 
file transfer server to launch the remote renderer and remote monitor. In such a case, it 
is preferable that the remote file transfer server, and not the remote file transfer client, 
handle writing the data to the sharable data and log format files. In addition, in such a 
25 case, the remote procedure call need not be the DCE pipe, because in real time, the data 
is accumulating more slowly and may be be transmitted at a slower rate. 

As described in co-pending U.S. Patent Application Serial Number 08/772,956 
and as illustrated in FIG.7A and FIG.7B, the transmitted data is received by the remote 

3Q file transfer client 440, de-crypted and unmarshalled using a de-cryption/un- 
marshalling module 580. The data is then put into a remote queue 600. Then the remote 
file transfer client encrypts, if desired, and marshalls the data using an encrytption and 
marshalling module 610 and uses a second remote procedure call 620 to send the data 
to the remote file transfer server. The remote file transfer server decrypts and un- 

35 marshalls the data using a de-cryption/un-marshalling module and decompresses it 
using a decompression module 640. Then using its own copy of the RWW module 490, 
the remote file transfer server writes the data to writes the data to the sharable remote 
data format file and/or the sharable remote log format file, and creates the related 
semaphore files 470, 480. 

40 
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As described in greater detail in co-pending U.S. Patent Application Serial 
Number 08/772,956, if it is desired to view the data as it is being written, the remote 
renderer can use its own copy of the RWW module 490 to read the data from the 
sharable log format file as it is being written and sends the data, through drivers, to the 
remote printer or remote monitor. This allows a person at the remote location to view 
or to print the data while it is being written to the remote sharable graphics data format 
66. 

The invention described in co-pending ILS. Patent Application Serial Number 
08/772,956 also provides a method of person-to-person communication at the same 
time the log transmission is occurring. 

The present invention can be used to transfer any type of file data and is 
especially useful in transmitting data as it is being acquired. 

One benefit of the present invention is that it allows files to be transferred 
securely, especially while "on the line." 

Another benefit of the present invention is that it allows files to be transferred 
while making maximum use of low bandwidth connections. 

Another benefit of the present invention is that it allows a file to be transferred 
while adaptably compressing the file to improve transmission throughput. 

Another benefit of the present invention is that it overcomes the disadvantages 
of the File Transfer Protocol. 

Another benefit of the present invention is that it allows files to be transferred 
taking into account the unique requirements of mobile network connections. 

Another benefit of the present invention is that it allows a file to be transferred 
as it is compiled in at least near real time from one locationlo a remote location remote 
from the primary for viewing or other use. 

Another benefit of the present invention is that it allows a well data file to be 
transferred as it is compiled in at least near real time from a wellsite to a remote location 
remote from the well site for viewing or other use. 

Another benefit of the present invention is that it provides a recovery method 
should communications be lost. 

Another benefit of the present invention is that it sends well data files from a 
wellsite to a remote location in near real time, in such a way that the data files are not 
susceptible to being misdirected or lost. 
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Another benefit of the present invention is that it can maintain the 
confidentiality of the well data while it is being transmitted. 

Another benefit of the present invention is that it allows transferring files in near 
real time from one location to a remote location so that so that persons can view the files 
in near real time, without the expense of travelling to the primary location. 

Another benefit of the present invention is that it allows transferring well data 
files in near real time from wellsite to a remote location remote from the wellsite so that 
persons can view the well data files near real time as they are being compiled, without 
the expense of travelling to the wellsite and without being exposed to the hazards of the 
wellsite. 

The principles, preferred embodiments, and modes of operation of the present 
invention have been described in the foregoing specification. The invention is not to be 
1§ construed as limited to the particular forms disclosed, because these are regarded as 
illustrative rather than restrictive. Moreover, variations and changes may be made by 
those skilled in the art without departing from the spirit of the invention. 

What is claimed is: 
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CLAIMS 

1 . A system for handling and transmitting a file having a degree of compressibility 
over a communication channel having a physical bandwidth wherein the improvement 
comprises; 

a. ) a file transfer server at a first location; 

b. ) a file transfer client at a second location; and\ 

c. ) a means for cop*fffessigg/ the file based on the physical bandwidth, the 
capabilities ouhe^tfansmitting and receiving processors and the degree of 
compressibility of the file. 



20 



2. A system for ma naging and transmitting a file having a degree of cartrjpressibility 
and a pU ggfity^ of bu ffer^ over a communication channel having a physical bandwidth 
1 5 whcreiTrtfie improvement comprises: 

a. ) a file transfer server at a first location; 

b. ) a file transfer client at a second location^and 

c. ) a feedback loop for optimally confessing the/file for transmission, including: 

1 . ) means for compressing a firsro«fifer%a first level of compressibility; 

2. ) means for evaluating the efficiency of the compxession^ofthe first buffer; 

3. ) means for adjusting the /^impression of second buffer ba§£a on the 



25 evaluation of the compression or the first buffer. 



3. A system for managing and transmitting a file having a plurality <6Ebuffers~oV«r a 
communication channel wherein the improvement comprises: 

a.) a means for compressing buffers of the file in a stream in real time while the 
30 file is being written. 

4. A system for managing and transmitting a file over a communication channel 
wherein the improvement comprises: 

a. ) a transmission of the file having a state and a location; and 

35 

b. ) a means for maintaining the state and location of the transmission within the 

file so that transmission can be resumed in the event of an interruption at the 
point of the interruption. 

40 
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5. A system as in claim 4 wherein the interrupted transmission can be automatically 
resumed. 

6. A system as in claim 4 wherein the interrupted transmission can be manually 
resumed. 

7. A system for managing and transmitting a file over a communication channel 
wherein the improvement comprises: \ 

a. ) a meaps^^^cryptin^ie file in a stream before and during transmission; and 

b. ) means for^K^crypting thg file after receipt of transmission. 

8. A system rouoatlaging and transmitting a file having a degree of compressibility 
and a plurality of buffers over a communication channel having a physical bandwidth 
wherein the improvement comprises: 

a. ) a transmitting processor; 

b. ) a receiving processor; 

c. ) a means for optimally compressing the file based on the physical bandwidth, 

the capabilities of the transmitting and receiving processors and the degree of 
compressibility of the file; 

d. ) a means for compressing the buffers of the file in a stream i 

the file is being written; 

e. ) a transmission of the file having a state and a location; 

f. ) a means for maintaining the state and location of the transmission within the 

file so that transmission can be resumed in the event of an interruption at the 
point of the interruption; 

g. ) a means for encrypting the file in a stream before and during transmission; and 

h. ) means for de-crypting the file after receipt of transmission. 

9. A inethod of compressing a source file, having a plurality of buffers, for 
transmission over a communication channel, comprising: 

a. ) selecting a first buffer of the source file; 

b. ) compressing the first buffer to a first compression level; 

c. ) marshalling the first buffer; 

d. ) transmitting the first buffer; 
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M r ^-~^mingthe firsTbuffer to a destination file; 

g. ) determining a first throughput which was used for steps (a) through (f); 

h. ) selecting the compression level of a second buffer based on the first 

throughput; and 

i. ) repeating steps (a) through (h) for each of the buffers in turn until all of the 

buffers of the source file have been transmitted and have been written to the 
destination file. 

10. A method as in claim 9, wherein the marshalling steps include encrypting the 
buffers and the de-compressing steps include de-encrypting the buffers. 

11. A method as in claim 1 0, further comprising the step of writing to the source file 
while performing one or more of steps (a) through (i). 

12. A system for handling and transmitting a first file over a communication channel 
comprising: 

a. ) a means for reading while writing data to the first file; 

b. ) a means for compressing the first file; 

c. ) a means for queueing the first file; 

d. ) a means for marshalling the first file; 

e. ) a means for transmitting the first file from a first location to a second location; 

f. ) a means for unmarshalling the first file; 

g. ) a means for decompressing the first file; and * 

h. ) a means for writing the first file to a second file. 

13. A system for handling and transmitting a first file as in claim 12 wherein the 
means for accomplishing steps a through d comprises a file transfer server. 

14. A system for handling and transmitting a first file as in claim 12 wherein the 
means for accomplishing steps f through g comprises a file transfer client. 

15. A system for handling and transmitting a first file as in claim 13 wherein the file 
transfer server includes a read while write module. 
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16. A system for handling and transmitting a first file as in claim 1 3 wherein the file 
transfer server includes a compression module. 

17. A system for handling and transmitting a first file as in claim 16 wherein the 
5 compression module uses a deflation algorithm. 

18. A system for handling and transmitting a first file as in claim 16 wherein the 
compression module uses a Huffman tree. 

19. A system for handling and transmitting a first file as in claim 16 wherein the 
compression module compresses in a first stage to produce literals and pointers. 

20. A system for handling and transmitting a first file as in claim 19 wherein the 
compression module compresses in a second stage to produce a Huffman tree and a 
block. 

21. A system for handling and transmitting a first file as in claim 12, further 
comprising 

a means for encrypting the first file before marshalling the first file and a 
means for decrypting the first file after the first file is transmitted. 

22. A system for handling and transmitting a first file as in claim 14 wherein the file 
transfer client includes a read while write module. 

23. A system for handling and transmitting a first file as in claim 2 1 , wherein the first 
file is a sharable file. 

24. A system for handling and transmitting a first file over a communication channel 
comprising: 

a. ) a means for reading while writing data to the first file; 

b. ) a means for compressing the first file; 

c. ) a means for queueing the first file; 

d. ) a means for marshalling the first file; 

e. ) a means for transmitting the first file from a first location to a second location; 

f. ) a first means for unmarshalling the first file; 

g. ) a means for re-queueing the first file; 

h. ) a means for re-marshalling the first file; 



15 



20 



30 



35 



40 



BNSDOCID: <WO 9828891 A 1_l_> 



WO 98/28891 PCT/US97/22688 

-26- 

i.) a remote procedure call for sending the first file within the second location; 
j.) a second means for unmarshalling the first file; 
k.) a means for decompressing the first file; and 
5 1.) a means for writing the first file to a second file. 

25. A system for handling and transmitting a first file as in claim 24 wherein the 
means for accomplishing steps a through d comprises a first file transfer server. 

10 26. A system for handling and transmitting a first file as in claim 24 wherein the 
means for accomplishing steps f through g comprises a file transfer client. 

27. A system for handling and transmitting a first file as in claim 25 wherein the file 
transfer server includes a read while write module. 

15 

28. A system for handling and transmitting a first file as in claim 25 wherein the file 
transfer server includes a compression module. 

29. A system for handling and transmitting a first file as in claim 28 wherein the 
compression module uses a deflation algorithm. 

20 

30. A system for handling and transmitting a first file as in claim 28 wherein the 
compression module uses a Huffman tree. 

31. A system for handling and transmitting a first file as in claim 28 wherein the 
25 compression module compresses in a first stage to produce literals and pointers. 

32. A system for handling and transmitting a first file as in claim 31 wherein the 
compression module compresses in a second stage to produce a Huffman tree and a 
block. 

30 

33. A system for handling and transmitting a first file as in claim 24, further 
comprising: 

a first means for encrypting the first file before marshalling the first file; and 
^ a first means for decrypting the first file after the first file is transmitted. 

34. A system for handling and transmitting a first file as in claim 33, further 
comprising: 

a second means for encrypting the first file after the first file is transmitted; 

40 
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a first means for decrypting the first file after the first file has been sent 
through the remote procedure call within the second location. 

35. A system for handling and transmitting a first file as in claim 24 wherein the 
means for accomplishing steps j through 1 comprises a second file transfer server. 

36. A system for handling and transmitting a first file as in claim 14 wherein the file 
transfer client includes a read while write module. \ 

37. A system for handling and transmitting a first file as in claim 35, wherein the first 
file is a sharable file. 

38. A method for handling and transmitting a first file over a communication channel 
comprising the steps of: 

a. ) reading while writing data to the first file; 

b. ) compressing the first file; 

c. ) queueing the first file; 

d. ) marshalling the first file; 

e. ) transmitting the first file from a first location to a second location; 
f ) unmarshalling the first file; 

g. ) decompressing the first file; and 

h. ) writing the first file to a second file. 

39. A method for handling and transmitting a first file as in claim 38 wherein steps a 
through d are accomplished by a file transfer server. 

40. A method for handling and transmitting a first file as in claim 3 8 wherein steps f 
through g are accomplished by a file transfer client. 

41 . A method for handling and transmitting a first file as in claim 39 wherein the file 
transfer server includes a read while write module. 

42. A method for handling and transmitting a first file as in claim 38 wherein the file 
transfer server includes a compression module. 

43. A method for handling and transmitting a first file as in claim 38 wherein the 
compression step includes use of a deflation algorithm for a first stage of compression. 
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44. A method for handling and transmitting a first file as in claim 38 wherein the 
compression step uses a Huffman tree for a second stage of compression. 

45. A method for handling and transmitting a first file as in claim 43 wherein the use 
of the deflation algorithm in the first stage produces literals and pointers. 

46. A method for handling and transmitting a first file as in claim 44 wherein the use 
in a second stage of the Huffman tree produces a Huffman tree and a block. 

47. A method for handling and transmitting a first file as in claim 38, further 
comprising the steps of: 

encrypting the first file before marshalling the first file; and 

decrypting the first file after the first file is transmitted. 

1 5 48. A system for handling and transmitting a first file as in claim 40 wherein the file 
transfer client includes a read while write module. 

49. A system for handling and transmitting a first file as in claim 38, wherein the first 
file is a sharable file. 

20 

50. A system for handling and transmitting a first file as in claim 38, wherein the 
second file is a sharable file. 

51. A method for handling and transmitting a first file over a communication channel 
2 5 comprising the steps of: 

a. ) reading while writing data to the first file; 

b. ) compressing the first file; 

c. ) queueing the first file; 

d. ) marshalling the first file; 

i 

e. ) transmitting the first file from a first location to a second location; 

f. ) unmarshalling the first file; 

g. ) re-queueing the first file; 

h. ) re-marshalling the first file; 

i. ) sending via a remote procedure call the first file within the second location; 
j.) unmarshalling the first file a second time; 

4 ^ k.) decompressing the first file; and 
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1.) writing the first file to a second file. 

52. A method for handling and transmitting a first file as in claim 5 1 wherein the steps 
a through d are accomplished by a first file transfer server. 

5 

53. A method for handling and transmitting a first file as in claim 51, wherein steps f 
through g are accomplished by a file transfer client. 

v . - ■ 

54. A method for handling and transmitting a first file as in claim 52 wherein the file 
transfer server includes a read while write module. 

10 

55. A method for handling and transmitting a first file as in claim 52 wherein the file 
transfer server includes a compression module. 

56. A method for handling and transmitting a first file as in claim 52 wherein the 
1 5 compression step includes use of a deflation algorithm for a first stage of compression. 

57. A method for handling and transmitting a first file as in claim 56 wherein the 
compression step uses a Huffman tree for a second stage of compression. 

20 58. A method for handling and transmitting a first file as in claim 57 wherein the use 
of the Huffman tree produces a Huffman tree and a block. 

59. A system for handling and transmitting a first file as in claim 56 wherein the use 
of the deflation algorithm in the first stage produces literals and pointers. 

25 

60. A method for handling and transmitting a first file as in claim 51, further 
comprising the steps of: 

encrypting the first file before marshalling the first file; and 

decrypting the first file after the first file is transmitted to the second location. 

30 

61. A system for handling and transmitting a first file as in claim 60, further 
comprising: 

encrypting the first file a second time, after the first file is transmitted; and 

35 decrypting the first file after the first file has been sent through the remote 

procedure call within the second location. 

62. A system for handling and transmitting a first file as in claim 51 , wherein steps j 
through 1 are accomplished using a second file transfer server. 

40 
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63. A system for handling and transmitting a first file as in claim 53, wherein the file 
transfer client includes a read while write module. 

64. A system for handling and transmitting a first file as in claim 61 , wherein the first 
5 file is a sharable file. 

65. A method of transmitting a file over a communication system using TCP/IP 
protocol comprising transmitting the file wherein the improvement comprises: 

recovering the file in the event of a communication interruption. 

10 

66. A system of transmitting a file over a communication system using TCP/IP 
protocol wherein the improvement comprises: 

a means for recovering the file in the event of a communication interruption. 

15 
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AMENDED CLAIMS 
[received by the International Bureau on 22 July 1998 (22.07.98); 
original claims 65 and 66 amended; new claims 67-72 added; remaining claims unchanged (3 pages)] 



63. A system for handling and transmitting a first file as in claim 53, wherein the file 
transfer client includes a read while write module. 

64. A system for handling and transmitting a first file as in claim 61 , wherein the first 
file is a sharable file. 

65. A system adapted for receiving a data file at a first location and for transmitting said 
data file from said first location to a second location via a communication link, 
comprising: 

a first apparatus at said first location adapted for receiving the data file and for modifying 
said data file thereby generating a modified data file; 

a sharable file at said first location, said sharable file including a semaphore file; 

a read-while-write apparatus at said first location operatively interconnected between the 
first apparatus and the sharable file for reading the modified <Jata file from the first 
apparatus while writing the modified data file to the sharable file; 

a primary file transfer server apparatus including a data compressor operatively 
connected to said sharable file and said semaphore file, s;aid primary file transfer server 
apparatus being adapted for determining if said semaphore file is associated with said 
sharable file and, when said semaphore file is associated with said sharable file, for 
reading said modified data file from said sharable file, while said modified data file is 
being written to said sharable file by said read-while-write apparatus, while writing said 
modified data file to said data compressor, 
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said data compressor of said primary file transfer server apparatus receiving said 
modified data file from said sharable file, compressing said modified data file, and 
providing the compressed modified data file to said communication link, the compressed 
modified data: file being transmitted to said second location via said communication link. 

66. The system of claim 65, wherein said data compressor comprises an adaptable data 
compressor adapted for performing adaptable compression of said modified data file. 

67. The system of claim 66, wherein said adaptable data compressor has a compression 
level and a throughput, and wherein, during said adaptable compression, said adaptable 
data compressor receives said throughput and changes said compression level of said 
data compressor in accordance with the received throughput. 

68. The system of claim 65, wherein said data compressor stores a deflation algorithm 
and executes the deflation algorithm during the compression of said modified data file. 

69. The system of claim 68, wherein said data compressor uses a Huffman tree after the 
execution of said deflation algorithm. 

70. The system of claim 67, wherein said primary file transfer server apparatus further 
includes an encryption marshalling module connected to said data compressor and 
adapted for encrypting said compressed modified data file and marshalling the encrypted 
compressed modified data file prior to transmitting the data file to said second location 
via said communication link. 

71. The system of claim 70, further comprising a decryption and un-rnarshalling module 
at said second location and operatively responsive to said data file transmitted to said 
second location via said communication link, a decompression module operatively 
connected to said decryption and un-marshalling module, a remote sharable file, and a 
further read-while-write apparatus operatively interconnected between said 
decompression module and said remote sharable file. 
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72. A method of transmitting a data file from a first location to a second location, 
comprising the steps of: 

receiving said data file in a first apparatus located at said first location; 

reading said data file from said first apparatus whileVriting said data file to a sharable 
file; 

reading said data file from said sharable file while writing said data file to a data 
compressor, 

setting a compression level in said data compressor in accordance with a throughput of 
said data compressor and compressing said data file at said compression level thereby 
generating a compressed data file; 

encrypting and marshalling said compressed data file; and 

transmitting the encrypted and marshalled and compressed data file to said second 
location. 
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STATEMENT UNDER ARTICLE 19 

The claims of this application have been amended by replacing originally filed claims 
65 and 66 with new replacement claims 65 and 66, and by adding new claims 67 
through 72. The originally filed claims 1 through 64 remain unchanged. 

The specification of this application discloses the following general concept: 

A data file is received in a first location and is transmitted from the first location to a 
second location via a communication link. A receiving apparatus (340, 350 in figure 
5 A), located at the first location, receives the data file, and a first read- while- write 
module (360 in figure 5 A) at the first location will read the data file from the receiving 
apparatus while writing the data file to a sharable file (370, 380). A further read- 
while-write apparatus (360 in figure 7 A) at the first location reads the data file from 
the sharable file, while it is being written to the sharable file by the first read while 
write apparatus (360), and writes the data file to a compression module (530 in figures 
6 and 7A). The compression module may include an "adaptable compression" feature 
(figure 10) wherein the compression level of the compression module varies depending 
on the throughput of the compression module; alternatively, the compression module 
may execute a deflation algorithm and it may use a Huffman tree. The data file is 
compressed in the compression module, and the compressed data file is encrypted and 
marshalled via the encryption module (550 in figure 6, 7A) prior to being transmitted 
from the first location to the second location via the communication link. 

No "new matter" is being introduced into new claims 65 through 72. As a result, the 
originally filed specification and drawings of this application remain unchanged. 
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